Dynamics of neural networks with non-monotone activation function
نویسندگان
چکیده
منابع مشابه
Chaotic Dynamics in Iterated Map Neural Networks with Piecewise Linear Activation Function
The paper examines the discrete-time dynamics of neuron models (of excitatory and inhibitory types) with piecewise linear activation functions, which are connected in a network. The properties of a pair of neurons (one excitatory and the other inhibitory) connected with each other, is studied in detail. Even such a simple system shows a rich variety of behavior, including high-period oscillatio...
متن کاملNonparametric regression using deep neural networks with ReLU activation function
Consider the multivariate nonparametric regression model. It is shown that estimators based on sparsely connected deep neural networks with ReLU activation function and properly chosen network architecture achieve the minimax rates of convergence (up to log n-factors) under a general composition assumption on the regression function. The framework includes many well-studied structural constrain...
متن کاملRecurrent Neural Networks with Iterated Function Systems Dynamics
We suggest a recurrent neural network (RNN) model with a recurrent part corresponding to iterative function systems (IFS) introduced by Barnsley [1] as a fractal image compression mechanism. The key idea is that 1) in our model we avoid learning the RNN state part by having non-trainable connections between the context and recurrent layers (this makes the training process less problematic and f...
متن کاملComplex-valued Neural Networks with Non-parametric Activation Functions
Complex-valued neural networks (CVNNs) are a powerful modeling tool for domains where data can be naturally interpreted in terms of complex numbers. However, several analytical properties of the complex domain (e.g., holomorphicity) make the design of CVNNs a more challenging task than their real counterpart. In this paper, we consider the problem of flexible activation functions (AFs) in the c...
متن کاملStability Analysis of Discrete Hopfield Neural Networks with the Nonnegative Definite Monotone Increasing Weight Function Matrix
The original Hopfield neural networks model is adapted so that the weights of the resulting network are time varying. In this paper, the Discrete Hopfield neural networks with weight function matrix DHNNWFM the weight changes with time, are considered, and the stability of DHNNWFM is analyzed. Combined with the Lyapunov function, we obtain some important results that if weight function matrix W...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Network: Computation in Neural Systems
سال: 1993
ISSN: 0954-898X,1361-6536
DOI: 10.1088/0954-898x_4_1_001